1. Field of Invention
This invention relates generally to network management. The invention is more specifically related to a presenting data about network operations and performance in a network administrator friendly format.
2. Discussion of Background
Networks have become important, strategic foundations for competing in a global economy. And every day, more users and more business activities step into the network realm. Users are demanding that access to the network be delivered with the same simplicity, reliability, quality, and flexibility associated with picking up a phone and calling anywhere in the world.
In this network-centric environment, the behind-the-scenes designers and operators of networks face three concurrent challenges:                Networks are growing larger and more dispersed. With each new user comes another PC, many of them notebooks and a growing number are scattered to remote locations, users' home offices, or all corners of the earth in the hands of mobile workers. While the initial capital cost of PCs gets a lot of attention, the greatest chunk of technology budgets goes to maintaining and managing users' connections. Lowering total cost of ownership (TCO) for a growing PC population means finding a way to simplify and automate management and troubleshooting of desktops, servers, and mobile PCs spread across the business. It also means enhancing the fault tolerance of servers so IS staffs escape the need to constantly fine-tune performance or deal with emergencies.        Networks are growing more complex. New technologies and applications are reaching into corporate networks at an accelerating pace. More users and greedier applications are competing for bandwidth. Fully switched local networks are common. Consequently, it is growing more difficult to monitor performance and pinpoint problems. The terminal-to-host model is long gone, and even the relatively predictable shape and behavior of client-server environments has given way to the more fluid and “fractal” connections that support intranets, Internet connectivity, multimedia, and other emerging applications. It is essential to gain increased visibility into today's complex networks.        Networks are growing more critical. Businesses are using information technology as a competitive advantage. And those that are getting the strongest return on their investment are graduating from e-mail and office applications to business-critical, production applications-process controls, customer service, medical imaging, e-commerce and others. In these situations, near-100-percent network availability is essential—the days of overnight outages lasting up to several hours are over.        
Along with this need for nonstop networks comes a need for greater control over, and information about, how traffic moves through the network. Companies are deploying applications such as enterprise resource planning (ERP), sales automation, e-commerce, distributed training, and voice-over-IP (VoIP) at a rapid pace. This business-critical and, in the case of voice and video, delay-sensitive traffic needs to move through the network unobstructed by lower-priority activities such as e-mail or Internet browsing.
Therefore, IS managers need tools capable of providing network support for these critical business objectives on an immediate real time basis. The supporting information is preferably provided in an intuitive format that allows the managers to quickly find the needed information with simple searching and a minimum of additional research. However, because of display complexities, and lack of available tools, the data presented to IS managers is not presented as effectively as is preferred to allow quick and easy access to the data needed for making proper network management decisions.
Various network monitoring tools currently utilized to inform managers about the health of networks are know. A brief discussion of network devices, standards, the traffic crossing these networks, and some an available tools is now provided.
Networking Devices Standards
This specification presumes familiarity with the general concepts, protocols, and devices currently used in LAN networking applications and in WAN internetworking applications. These standards are publicly available and discussed in more detail in the above referenced and other co-assigned patent applications.
This specification also presumes some familiarity with the specific network and operating system components discussed briefly in the following paragraphs, such as the simple network management protocol (SNMP) for management of LAN and WAN networks, and the RMON MIBs defined for remote network monitoring and management. For additional information regarding MIBs, the reader is referred to RFC 1213.
General Network Topology
Local area networks (LANs) are arrangements of various hardware and software elements that operate together to allow a number of digital devices to exchange data within the LAN and also may include internet connections to external wide area networks (WANs). FIG. 1 is a diagram representing a typical modern LAN 100 comprised of one to many Hosts or End Systems (ESs) such as hosts 1 . . . n that are responsible for data transmission throughout the LAN. The ESs may be familiar end-user data processing equipment such as personal computers, workstations, and printers and additionally may be digital devices such as digital telephones or real-time video displays. Different types of ESs can operate together on the same LAN. In FIG. 1, the hosts 1 . . . n are on a ring type LAN having a server/bridge/router device 130, also referred to as an Intermediate System (IS), that serves various network functions for the LAN 100, and administers each of a bridge to an attached LAN 160 and a router to a Wide Area Network (WAN) 190. However, modern networks may be composed of any number of hosts, bridges, switches, hubs, routers, and other network devices, and may be configured as any one or more of ring, star, and other configurations.
Packets
In a LAN such as 100, data is generally transmitted between ESs as independent packets, with each packet containing a header having at least a destination address specifying an ultimate destination and generally also having a source address and other transmission information such as transmission priority. Packets are generally formatted according to a particular protocol and contain a protocol identifier of that protocol. Packets may be encased in other packets.
Layers
Modern communication standards, such as the TCP/IP Suite and the IEEE 802 standards, organize the tasks necessary for data communication into layers. At different layers, data is viewed and organized differently, different protocols are followed, different packets are defined and different physical devices and software modules handle the data traffic. FIG. 2 illustrates various examples of layered network standards having a number of layers. Corresponding levels of the various network standards are shown adjacent to each other and the OSI layers, which are referred to herein as: the Physical Layer, the Data Link Layer, the Routing Layer, the Transport Layer, the Session Layer, the Presentation Layer and the Application Layer, are also shown for reference. Please note the TCP/IP protocol layers shown.
Generally, when an ES is communicating over a network using a layered protocol, a different software module may be running on the ES at each of the different layers in order to handle network functions at that layer.
Drivers and Adapters
Each of the ISs and ESs in FIG. 1 includes one or more adapters and a set of drivers. An adaptor generally includes circuitry and connectors for communication over a segment and translates data from the digital form used by the computer circuitry in the IS or ES into a form that may be transmitted over the segment, which may be electrical signals, optical signals, radio waves, etc. A driver is a set of instructions resident on a device that allows the device to accomplish various tasks as defined by different network protocols. Drivers are generally software programs stored on the ISs or ESs in a manner that allows the drivers to be modified without modifying the IS or ES hardware.
NIC Driver
The lowest layer adaptor software operating in one type of network ES is generally referred to as a NIC (Network Interface Card) driver. A NIC driver is layer 2 software designed to be tightly coupled to and integrated with the adaptor hardware at the adaptor interface (layer 1) and is also designed to provide a standardized interface between layer 2 and 3. Ideally, NIC drivers are small and are designed so that even in an ES with a large amount of installed network software, new adaptor hardware can be substituted with a new NIC driver, and all other ES software can continue to access the network without modification.
NIC drivers communicate through one of several available NIC driver interfaces to higher layer network protocols. Examples of NIC driver interface specifications are NDIS (Network Driver Interface Specification developed by Microsoft and 3Com) and ODI (Open Data-Link Interface developed by Apple Computer and Novell).
Generally, when an ES is booting up and begins building its stack of network protocol software, the NIC driver loads first and tends to be more robust than other network software modules because of its limited functions and because it is tightly designed to work with a particular hardware adaptor.
Management and Monitoring of ESs
A network such as that shown in FIG. 1 is generally managed and monitored within an enterprise by an Information Services (IS) Department (ISD), and an IS manager, which is responsible for handling all the interconnections and devices shown. The same ISD is generally responsible for managing the applications and system components on each of the individual ESs in the network.
Many prior art systems have been proposed to allow an IS staff person to manage and partially monitor network infrastructure remotely over a network. Such systems include IBM's NetView, HP's OpenView or Novell's Network Management System (NMS). However, these systems generally rely on a full network protocol stack to be correctly running effectively on the remote ES in order to accomplish any remote file management operations.
Simple Network Management Protocol (SNMP)
A common protocol used for managing network infrastructure over the network is the Simple Network Management Protocol (SNMP). SNMP is a layer 7 network and system management protocol that handles network and system management functions and can be implemented as a driver (or SNMP agent) interfacing through UDP or some other layer 4 protocol. Prior art SNMP installations largely were not placed in ESs because SNMP did not handle ES management or monitoring functions and because SNMP agents are processor and memory intensive.
SNMP is designed to provide a simple but powerful cross platform protocol for communicating complex data structures important to network infrastructure management. However, its power and platform-independent design makes it computationally intensive to implement, and for that reason it has limited applications in end system management or monitoring. It is primarily used in network infrastructure management, such as management of network routers and bridges.
SNMP is designed to support the exchange of Management Information Base (MIB) objects through use of two simple verbs, get and set. MIB objects can be control structures, such as a retry counter in an adaptor. Get can get the current value of the MIB and set can change it. While the SNMP protocol is simple, the MIB definitions can be difficult to implement because MIB ids use complex data structures which create cross-platform complexities. SNMP has to translate these complex MIB definitions into ASN.1 which is a cross-platform language.
Even if installed in an ES, an SNMP agent cannot be used to manage or diagnose an ES or update system components where the UDP protocol stack is not working properly, which will often be the case when the network connection is failing. When working, SNMP provides a protocol interface for higher layer prior art management applications.
SNMP is described in detail in a number of standard reference works. The wide adoption of SNMP throughout the networking industry has made compatibility with SNMP an important aspect of new management and monitoring tools. For additional information regarding SNMP, the reader is refferred to RFC 1905.
RMON Overview
Remote Monitoring (RMON) technology is a set of software and hardware specifications designed to facilitate the monitoring and reporting of data traffic statistics in a local area network (LAN) or wide area network (WAN). RMON was originally defined by the IETF (Internet Engineering Task Force) in 1991. RMON defined an independent network probe, which was generally implemented as a separate CPU-based system residing on the monitored network. Software running on the probe and associated machines provided the various functions described by the defining IETF RFC documents, RFC-1271, RFC-1513 and RFC-1757.
According to the original standards, a special application program, sometimes referred to as an RMON Manager, controlled the operation of the probe and collected the statistics and data captured by the probe. In order to track network traffic and perform commands issued to it by the RMON Manager, a prior art probe operated in a promiscuous mode, where it read every packet transmitted on network segments to which it was connected. The probe performed analyses or stored packets as requested by the RMON Manager.
RMON builds upon the earlier Simple Network Management Protocol (SNMP) technology while offering four advantages over SNMP agent-based solutions:
(1) RMON provides autonomous Network Management/Monitoring, unlike SNMP which required periodic polling of ESs. RMON stand-alone probes are constantly on duty and only require communication with a management application when a user wishes to access information kept at the probe.
(2) RMON's alarm capability and user-programmable event triggers furnish a user with asynchronous notification of network events without polling ESs. This reduces the network bandwidth used and allows across-WAN links without concern for performance costs.
(3) RMON automatically tracks network traffic volume and errors for each ES MAC address seen on a segment and maintains a Host Matrix table of MAC address pairs that have exchanged packets and the traffic volume and errors associated with those address pairs.
(4) RMON permits the collection and maintenance of historical network performance metrics thereby facilitating trend analysis and proactive performance monitoring.
(5) RMON includes fairly sophisticated packet filter and capture capabilities which allowed a user to collect important network packet exchanges and analyze them at the management console.
The new capabilities of RMON were quickly appreciated and RMON probes soon became the preferred choice for remote monitoring. It has become common place for ISs, particularly hubs and switch/bridges to embed RMON probe functions.
For additional information regarding RMON, the reader is referred to RFC's 2021, 2819, 2895, and 1902.
RMON2
Shortly after adoption of RMON, users wanted more management information than the layer 2 statistics RMON provided. In particular, network managers wanted to track higher layer protocols and the sessions based upon those protocols to learn which applications were using which protocols at what expense in available network bandwidth. Therefore, a new version of RMON, RMON2 was developed to provide more advanced capabilities. RMON2 provides network header layer (layer 3) through application layer (layer 7) monitoring for a number of commonly used protocols and applications, including the Internet protocol suite (IP and UDP) and Internet applications (FTP, Telnet, TCP and SNMP).
Limitations of IS-Based (Hub-Based/Switch-Based RMON)
A traditional stand-alone RMON probe, connected to a switch like any other host device, only sees network traffic flowing on the segments to which it is connected, greatly limiting its usefulness in modern, more complicated network topologies. One solution is to place the RMON probe within the switch itself and have it monitor all ports simultaneously. However, this requires considerable processing capability in order to handle the large bandwidth made possible by modern switching architectures.
In a conventional 10 Mb Ethernet or 4/16 Mb Token Ring environment, a stand-alone RMON probe on a single network segment could usually be implemented on a 486-class processor. However, where multiple network interfaces must be monitored or where network bandwidths are higher, (such as with 100Base-T LANs or switching hubs/ATM), it is considerably more costly to build a probe with sufficient processing power to capture all, or even most, of the network packets being exchanged. Independent laboratory tests show that RMON products claiming to keep up with higher bandwidth network traffic generally cannot, in fact, keep up with all data flow during peak network rates. The situation worsens considerably when attempting to do RMON2 analysis of network packets in high bandwidth environments. Processing power required can be easily five times greater than needed to simply capture packets, and data storage requirements can easily increase ten fold.
Use of filtering switches and hubs (discussed in the above referenced patent applications) in networks further limits the usefulness of probes because, unlike repeaters, not all the packets appear at every output port of the switch. This makes the use of external stand-alone probes infeasible unless the switch vendor has provided a monitor port (sometimes called a copy port) where all packets are repeated to the external RMON probe. However, this approach decreases data traffic performance in the switch, and does nothing to reduce the processing overhead required of the probe.
In general, the RMON and RMON2 standards fail to provide all the information needed by IS managers for efficient network management. This functionality would need to be implemented in a LAN/WAN without unduly harming network performance and not requiring additional expensive network hardware to support. In addition, network data needs to be presented to the IS managers in an efficient manner.